Explaining artificial neural network ensembles: A case study with electrocardiograms from chest pain patients

Artificial neural networks is one of the most commonly used machine learning algorithms in medical applications. However, they are still not used in practice in the clinics partly due to their lack of explanatory capacity. We compare two case-based explanation methods to two trained physicians on analysis of electrocardiogram (ECG) data from patients with a suspected acute coronary syndrome (ACS). The median overlaps of the top 5 selected features between the two physicians, and a given physician and a method, were initially low. Using a correlation analysis of the features the median overlap increased to values typically in the range 2-3. In conclusion, both our case-based methods generate explanations somewhat similar to those of trained expert physicians on the problem of diagnosing ACS from ECG data. Appearing in the Proceedings of the ICML/UAI/COLT 2008 Workshop on Machine Learning for Health-Care Applications, Helsinki, Finland, 2008. Copyright 2008 by the author(s)/owner(s). 1. Background Artificial neural networks (ANN) has been gaining interest in the medical community for quite some time now, and has proven useful for many clinical decision problems (Harrison & Kennedy, 2005; Goldman et al., 1996; Baxt et al., 2002; Green et al., 2006; Green et al., 2005; Kennedy & Harrison, 2006; Lisboa, 2002). Still, as of today, there are very few live applications in use at the clinics. Though the reasons for this low usage are numerous (Bates et al., 2003), one major drawback is the lack of interpretability of the decisions provided by an ANN (Lisboa, 2002). Most efforts of making sense out of an ANN decision is based on rule extraction methods where the decision boundary is discretized into segments. There are basically two ways of attacking this problem in neural networks. The first is the decompositional (Kolman & Margaliot, 2005) approach where the network is scrutinized from within in order to extract useful information about a decision. This is usually done by analyzing the activations of individual nodes in the network as well as the weights leading into them. This methodology was used by (Kolman & Margaliot, 2005) where they demonstrated that an ANN is mathematically equivalent to an all permutation fuzzy rule base. Their work provided an explicit way of transforming an ANN into a set of IF THEN rules. Despite beExplaining artificial neural network ensembles ing intuitively attractive this approach lead to a large number of rules that had to be reduced. The second one known as the pedagogical (Saad & Wunsch, 2007; Etchells & Lisboa, 2006) approach treats the network as a black box. Here the analysis is based on examining the relationship between what is fed into the network with what is returned as output. In a recent paper by (Etchells & Lisboa, 2006) the pedagogical approach was used when developing the orthogonal search based rule extraction (OSRE) method that successfully extracted the exact rules for the Monks (Thrun et al., 1991) data. They also point out that, in the presence of large node output weights, the decompositional approach may fail to accurately describe the logic of the network. Another way to analyze a neural network is by sensitivity analysis where the main focus has been on extracting global properties. Usually this has been accomplished by analyzing the weights in the network on a pattern by pattern basis. Interestingly enough this has been considered a drawback by several authors (Montaño & Palmer, 2003; Tchaban et al., 1998; Wang et al., 2004). From a medical application point of view it is often necessary to provide an explanation underlying a given decision. If the decision support is to function in a stressful clinical setting (e.g. an emergency department) then it is required to provide a fast explanation for each case, easily interpretable by the operator. This case-based feed-back requirement is lacking in most methods for analyzing the operation of a neural network ensemble. We believe this has severely limited the full potential of using neural networks in a clinical decision support system. The idea of using the specific case at hand as the basis for the feed-back algorithm is not new. In (Haraldsson et al., 2004) a specific method was developed for electrocardiogram curves, where the case-based feed-back was presented as modified curves representing changes towards being more healthy or non-healthy. In (Wall et al., 2003) rules were extracted and later ranked depending on the prediction of the case. The idea was that more complex rules should be presented when the decision support system classified a patient as healthy. Conversely if a patient were classified as non-healthy, less complex rules were given as feed-back. Another approach to case-based explanation can be found in (Caruana, 2000) where the reasoning behind the neural network was presented as showing a set of similar cases. When providing feedback to a physician in a clinical situation we need to make sure that only the core of the driving forces behind a classification is presented. This means that a rule based approach, where possibly more than 10 rules are presented per case, will be difficult to use in practice. Also many of the rules will be non-specific for a given case since the rules are extracted globally from the data set with the aim of approximating the decision boundary of the ANN. To us this suggests that any case-based feedback should be derived from a single case and not the entire data set. Case-based feed back is indeed dependent on the question one is asking. In a clinical setting we often find the important feed-back to simply be the set of variables, most important for the decision. The two approaches described in this study will both result in a ranked list of important variables and the explanation will simply consist of the topmost important ones, for each case. In this work a case study was performed where we explored the explanatory power of an ANN ensemble in the context of predicting acute coronary syndromes, in chest pain patients, from electrocardiogram (ECG) data alone. Even though we only investigated this particular medical application, we still believe that the results are transferable to many other medical problems as well.